Dependable Multimodal Communication and Interaction with Robotic Assistants
نویسندگان
چکیده
To advance research in the field of multimodal human-robot communication we designed and built the humanoid robot HERMES. Equipped with an omnidirectional undercarriage and two manipulator arms it combines visual, kinesthetic, tactile, and auditory sensing with natural spoken language input and output and body expressions for natural communication and interaction with humans. HERMES was successfully tested in an extended six-month experiment in a museum where only naive users interacted with the robot. They chatted with HERMES in several languages and requested various services. Multimodal communication and an understanding of the current situation by the robot turned out to be the key to success.
منابع مشابه
Achieving Multimodal Cohesion during Intercultural Conversations
How do English as a lingua franca (ELF) speakers achieve multimodal cohesion on the basis of their specific interests and cultural backgrounds? From a dialogic and collaborative view of communication, this study focuses on how verbal and nonverbal modes cohere together during intercultural conversations. The data include approximately 160-minute transcribed video recordings of ELF interactions ...
متن کاملA multi-modal dialog system for a mobile robot
A challenging domain for dialog systems is their use for the communication with robotic assistants. In contrast to the classical use of spoken language for information retrieval, on a mobile robot multimodal dialogs and the dynamic interaction of the robot system with its environment have to be considered. In this paper we will present the dialog system developed for BIRON – the Bielefeld Robot...
متن کاملMultimodal Resources for Human-Robot Communication Modelling
This paper reports on work related to the modelling of Human-Robot Communication on the basis of multimodal and multisensory human behaviour analysis. A primary focus in this framework of analysis is the definition of semantics of human actions in interaction, their capture and their representation in terms of behavioural patterns that, in turn, feed a multimodal human-robot communication syste...
متن کاملTowards Effective Communication with Robotic Assistants for the Elderly: Integrating Speech, Vision and Haptics
Our goal is to develop an interface for older people to effectively communicate with a robotic assistant so that they can safely remain living in their home. We are devising a multimodal interface since people communicate with one another using a variety of verbal and non-verbal signals, including haptics, i.e., physical interactions. We view haptics as an integral component of communication, w...
متن کاملModeling and Solving Human-Robot Collaborative Tasks Using POMDPs
Robots have helped in the automation of repetitive tasks. However, they have limited roles as co-workers or assistants to humans because of their limited sensing and perception abilities, which in turn leads to limited inference capabilities. To help with a task a robot can learn to infer its human user’s state in a process, which is hidden information. This inference is multimodal and over a l...
متن کامل